Query Fresh: Log Shipping on Steroids

Hot standby systems often have to trade safety (i.e., not losing committed work) and freshness (i.e., having access to recent updates) for performance. Guaranteeing safety requires synchronous log shipping that blocks the primary until the log records are durably replicated in one or multiple backups; maintaining freshness necessitates fast log replay on backups, but is often defeated by the dual-copy architecture and serial replay: a backup must generate the “real” data from the log to make recent updates accessible to read-only queries. This paper proposes Query Fresh, a hot standby system that provides both safety and freshness while maintaining high performance on the primary. The crux is an append-only storage architecture used in conjunction with fast networks (e.g., InfiniBand) and byteaddressable, non-volatile memory (NVRAM). Query Fresh avoids the dual-copy design and treats the log as the database, enabling lightweight, parallel log replay that does not block the primary. Experimental results using the TPC-C benchmark show that under Query Fresh, backup servers can replay log records faster than they are generated by the primary server, using one quarter of the available compute resources. With a 56Gbps network, Query Fresh can support up to 4–5 synchronous replicas, each of which receives and replays ∼1.4GB of log records per second, with up to 4–6% overhead on the primary compared to a standalone server that achieves 620kTPS without replication. PVLDB Reference Format: Tianzheng Wang, Ryan Johnson, Ippokratis Pandis. Query Fresh: Log Shipping on Steroids. PVLDB, 11(4): 406 419, 2017. DOI: https://doi.org/10.1145/3164135.3164137